CME 305 Project: Learning matrices

A foundational concept in modern machine learning is to construct models from data by balancing the complexity of the model with the discrepancy between the model and the data. Therefore, one of the main concerns is to define meaningful ways to quantify complexity and discrepancy. To formulate an abstract framework, consider a class F of models, endowed with a set of operations on its elements. We define a function C : F 7→ R+ that quantifies the complexity of these models. Any model with low values of C is deemed simple in this class. Furthermore, we assume that F is spanned by the simple models induced by C. In addition, consider a function DY : F 7→ R+ that measures the discrepancy between the data Y and any given model. In other words, any model with high values of DY cannot explain properly the data. Thus the problem reduces to determine reasonable choices of C and DY for a given application. Once these are made, one common scheme to construct the aforementioned model is to solve an optimization problem. Usually, this problem has the form minimize X∈F DY (X) subject to C(X) ∈ S (1)